Feature/covalent #196
Feature/covalent #196
Conversation
satyaog
force-pushed
the
feature/covalent
branch
5 times, most recently
from
53c00e3
to
ac6b421
Compare
satyaog
force-pushed
the
feature/covalent
branch
3 times, most recently
from
dcad77a
to
90cad06
Compare
satyaog
force-pushed
the
feature/covalent
branch
from
90cad06
to
5e2f347
Compare
satyaog
force-pushed
the
feature/covalent
branch
3 times, most recently
from
f6fc50d
to
ba6f280
Compare
satyaog
force-pushed
the
feature/covalent
branch
from
ba6f280
to
b06ed6a
Compare
covalent is not compatible with milabench as it requires sqlalchemy<2.0.0
satyaog
force-pushed
the
feature/covalent
branch
from
b06ed6a
to
c3b9dbf
Compare
.github/workflows/cloud-ci.yml
Outdated
| with: | ||
| token: ${{ secrets.REPORTS_PAT }} |
There was a problem hiding this comment.
I failed to make this work to be able to push to the reports branch
| with: | |
| token: ${{ secrets.REPORTS_PAT }} |
.github/workflows/cloud-ci.yml
Outdated
|
|
||
| - name: Summary | ||
| run: | | ||
| git remote set-url origin "https://${{ vars.REPORTS_USERNAME }}:${{ secrets.REPORTS_PAT }}@$(git remote get-url origin | cut -d'/' -f3-)" |
There was a problem hiding this comment.
There's probably a better way to do this with actions/checkout@v3
milabench/cli/covalent/__main__.py
Outdated
| **_get_executor_kwargs(args), | ||
| ) | ||
|
|
||
| def _popen(cmd, *args, _env=None, **kwargs): |
There was a problem hiding this comment.
This should probably be simplified, I don't think we'll use the complex features on configs and args paths
milabench/common.py
Outdated
| reports_url = ([ | ||
| _r.url for _r in _repo.remotes if "mila-iqia" in _r.url | ||
| ] or [ | ||
| _r.url for _r in _repo.remotes if _r.name == "origin" | ||
| ])[0] | ||
| reports_url = XPath("github.com".join(reports_url.split("github.com")[1:])[1:]) | ||
| reports_url = XPath("https://github.com") / f"{reports_url.with_suffix('')}/tree/{reports_repo.active_branch.name}" |
There was a problem hiding this comment.
This might not be needed if we set the link with markdown instead of the svg itself to redirect to the summary page
milabench/common.py
Outdated
| tag = ([ | ||
| t.name | ||
| for t in _repo.tags | ||
| if meta[0]["milabench"]["tag"].startswith(t.name) | ||
| ] or [meta[0]["milabench"]["tag"]])[0] |
There was a problem hiding this comment.
I kind of feel that using the tag as is would be better even if there's a chance I think it will duplicate the versions? Let me know
| tag = ([ | |
| t.name | |
| for t in _repo.tags | |
| if meta[0]["milabench"]["tag"].startswith(t.name) | |
| ] or [meta[0]["milabench"]["tag"]])[0] | |
| tag = meta[0]["milabench"]["tag"] |
There was a problem hiding this comment.
There is this
I think it would be clearer if we use "branch-commit" because the tag means actually nothing in most cases.
For example, in master the latest tag is still v0.0.6 but stable latest tag is v0.0.10.
satyaog
force-pushed
the
feature/covalent
branch
5 times, most recently
from
c486705
to
1991c52
Compare
satyaog
force-pushed
the
feature/covalent
branch
3 times, most recently
from
f49c60a
to
0cb1bcd
Compare
|
The workflow here cannot access repo's secrest even if I'm an admin. Seams related to this explanation https://stackoverflow.com/a/58740879. Closing in favour of #217 which is a local branch |
milabench cloud --setupIt creates a system config file and takes a target cloud platform with
--run-on.This starts a local covalent server which is used to manage python code that will be executed on the remote. For now this is only somewhat useful since milabench is mostly using ssh commands anyway and it would take a bit of time to refactor the pipeline I think to instead use the covalent interface to run code. I think this could be an interesting approach but it's a nice to have for now.
So
milabench cloud --setupsetup the remote and install basic stuff on it like the correct python version (necessary to ensure good serialization/deserialization of python objects between the local and remote machine), pip and venv. venv is used to separate the covalent env and milabench env which have incompatible package requirements versions (sqlalchemy caused problems). On this is done , the covalent server becomes uselessThen system config file should be used in the install, prepare and run commands. In those commands it creates a new standalone config for the tests that will be executed and copies it to the remote before the rest of the pipeline is executed.
At the end of the run command the results are copied to the local machine to allow the generation of a report
At the very end,
milabench cloud --teardownshould be used to release the cloud resources. The--allargument will release all resources of a target cloud platform specified with--run-on.milabench report --pushPush the results to a
reportsbranch which as well stores the status svg and summaryExample of reports : #210